We construct near-optimal coresets for kernel density estimate for points in \mathbb{R^d when the kernel is positive definite. Specifically we show a polynomial time construction for a coreset of size O( \sqrt{d\log (1/\epsilon) / \epsilon ), and we show a near-matching lower bound of size \Omega(  d/ \epsilon). The upper bound is a polynomial in 1/ \epsilon improvement when d \in [3,1/ \epsilon^2) (for all kernels except the Gaussian kernel which had a previous upper bound of O((1/\epsilon) \log^d (1/\epsilon)) ) and the lower bound is the first known lower bound to depend on d for this problem. Moreover, the upper bound restriction that the kernel is positive definite is significant in that it applies to a wide-variety of kernels, specifically those most important for machine learning. This includes kernels for information distances and the sinc kernel which can be negative.
We construct near-optimal coresets for kernel density estimates for points in \mathbb{R when the kernel is positive definite. Specifically we show a polynomial time construction for a coreset of size O( \sqrt{d / \varepsilon\cdot \sqrt{\log 1/\varepsilon ), and we show a near-matching lower bound of size \Omega( \min\{ d/ \varepsilon, 1/\varepsilon^2\}). When d\geq 1/\varepsilon^2, it is known that the size of coreset can be O(1/\varepsilon^2). The upper bound is a polynomial-in-( 1/ \varepsilon) improvement when d \in [3,1/ \varepsilon^2 ) and the lower bound is the first known lower bound to depend on d for this problem. Moreover, the upper bound restriction that the kernel is positive definite is significant in that it applies to a wide-variety of kernels, specifically those most important for machine learning. This includes kernels for information distances and the sinc kernel which can be negative.
R
estimate
estimates
R
\mathbb{R^d
\mathbb{R
R
\sqrt{d\log (1/\epsilon)
\sqrt{d
R
\epsilon
\varepsilon\cdot \sqrt{\log 1/\varepsilon
A
\min\{
R
\epsilon).
\varepsilon, 1/\varepsilon^2\}). When d\geq 1/\varepsilon^2, it is known that the size of coreset can be O(1/\varepsilon^2).
R
polynomial in
polynomial-in-(
R
\epsilon
\varepsilon)
R
\epsilon^2) (for all kernels except the Gaussian kernel which had a previous upper bound of O((1/\epsilon) \log^d (1/\epsilon))
\varepsilon^2