Random feature methods have been successful in various machine learning tasks, are easy to compute, and come with theoretical accuracy bounds. They serve as an alternative approach to standard neural networks since they can represent similar function spaces without a costly training phase. However, for accuracy, random feature methods require more measurements than trainable parameters, limiting their use for data-scarce applications or problems in scientific machine learning. This paper introduces the sparse random feature method that learns parsimonious random feature models utilizing techniques from compressive sensing . We provide uniform bounds on the approximation error  for functions in a  reproducing kernel Hilbert space  depending on the number of samples and the distribution of features. The error bounds improve with additional structural conditions, such as coordinate sparsity, compact clusters of the spectrum, or rapid spectral decay.  We show that the sparse random feature method outperforms shallow networks for well-structured functions and applications to scientific machine learning tasks.
Random feature methods have been successful in various machine learning tasks, are easy to compute, and come with theoretical accuracy bounds. They serve as an alternative approach to standard neural networks since they can represent similar function spaces without a costly training phase. However, for accuracy, random feature methods require more measurements than trainable parameters, limiting their use for data-scarce applications or problems in scientific machine learning. This paper introduces the sparse random feature expansion to obtain parsimonious random feature models . Specifically, we leverage ideas from compressive sensing to generate random feature expansions with theoretical guarantees even in the data-scarce setting. In particular, we provide uniform bounds on the approximation error and generalization bounds for functions in a certain class (that is dense in a reproducing kernel Hilbert space ) depending on the number of samples and the distribution of features. The error bounds improve with additional structural conditions, such as coordinate sparsity, compact clusters of the spectrum, or rapid spectral decay. In particular, by introducing sparse features, i.e. features with random sparse weights, we provide improved bounds for low order functions. We show that the sparse random feature expansions outperforms shallow networks in several scientific machine learning tasks.
R
method that learns
expansion to obtain
R
utilizing techniques
. Specifically, we leverage ideas
R
. We
to generate random feature expansions with theoretical guarantees even in the data-scarce setting. In particular, we
A
and generalization bounds
A
certain class (that is dense in a
A
)
A
In particular, by introducing sparse features, i.e. features with random sparse weights, we provide improved bounds for low order functions.
R
method
expansions
R
for well-structured functions and applications to
in several