We introduce kernel thinning, a simple algorithm for generating better-than-Monte-Carlo approximations to distributions \mathbb{P a suitable reproducing kernel k , and O(n^2) time, kernel thinning returns \sqrt{n in the associated reproducing kernel Hilbert space. With high probability, the maximum discrepancy in integration error is O_d(n^{-1{2}}\log n) for compactly supported P and O_d(n^{-1{2}} (\log n)^{d+1\log\log n}) for sub-exponential \mathbb{P}  . In contrast, an equal-sized i.i.d. sample from \mathbb{P} suffers \Omega(n^{-\frac14}) integration error. Our sub-exponential guarantees resemble the classical quasi-Monte Carlo error rates for uniform \mathbb{P} on [0,1]^d but apply to general distributions on \mathbb{R}^d and a wide range of common kernels. We use our results to derive explicit non-asymptotic maximum mean discrepancy bounds for Gaussian, Mat\'ern, and B-spline kernels and present two vignettes illustrating the practical benefits of kernel thinning over i.i.d. sampling and standard Markov chain Monte Carlo thinning.
We introduce kernel thinning, a new procedure for compressing a distribution \mathbb{P a suitable reproducing kernel k and O(n^2) time, kernel thinning compresses an n-point approximation to \mathbb{P in the associated reproducing kernel Hilbert space. With high probability, the maximum discrepancy in integration error is O_d(n^{-1{2}}\log n) for compactly supported P and O_d(n^{-1{2}} (\log n)^{d+1\log\log n}) for sub-exponential \mathbb{P} on \mathbb{R . In contrast, an equal-sized i.i.d. sample from \mathbb{P} suffers \Omega(n^{-\frac14}) integration error. Our sub-exponential guarantees resemble the classical quasi-Monte Carlo error rates for uniform \mathbb{P} on [0,1]^d but apply to general distributions on \mathbb{R}^d and a wide range of common kernels. We use our results to derive explicit non-asymptotic maximum mean discrepancy bounds for Gaussian, Mat\'ern, and B-spline kernels and present two vignettes illustrating the practical benefits of kernel thinning over i.i.d. sampling and standard Markov chain Monte Carlo thinning.
R
simple algorithm for generating better-than-Monte-Carlo approximations to distributions \mathbb{P
new procedure for compressing a distribution \mathbb{P
D
,
R
returns \sqrt{n
compresses an n-point approximation to \mathbb{P
A
on \mathbb{R