Improved Coresets for Euclidean k -Means
Abstract
Given a set of n points in d dimensions, the Euclidean k -means (resp.
Euclidean kmedian) problem consists of finding k centers such that the sum of squared distances (resp.
sum of distances)
from every point to its closest center is minimized.
The arguably most popular way of dealing with this problem in the big data setting is to first compress the data by computing a weighted subset known as a coreset and then run any algorithm on this subset.
The guarantee of the coreset is that for any candidate solution, the ratio between coreset cost and the cost of the original instance is less than a (1 ± ε ) factor.
The current state of the art coreset size is ˜
O (min( k 2 · ε − 2 , k · ε − 4 )) for Euclidean k -means and ˜ O
(min( k 2 · ε − 2 , k · ε − 3 )) for
Euclidean k -median.
The best known lower bound for both problems is Ω( kε − 2 ) .
In this paper, we improve these bounds to ˜ O
(min( k 3 / 2 · ε − 2 , k · ε − 4 )) for Euclidean k -means and ˜ O
(min( k 4 / 3 · ε − 2 , k · ε − 4 )) for Euclidean k -median.
In particular, ours is the first provable bound that breaks through the k 2 barrier while retaining an optimal dependency on ε .
Introduction
Coresets have become a staple in the design of algorithms for large data sets.
In the most general setting, a coreset compresses the data set in such a way that for any set of previously specified candidate queries, the cost of evaluating the query and the cost of the coreset are similar, up to an arbitrary small distortion.
A popular subject in coreset literature is the Euclidean k -means problem.
Here, we are given n points P in d dimensions and our task is to find a set of k points C called centers minimizing cost ( P, C ) :
= (cid:80)
p ∈
P min c ∈ C ∥
p − c ∥ 2 , where ∥ p − c ∥ =
(cid:113)(cid:80) di
=1 ( p i − c i ) 2 denotes the Euclidean distance.
In this case, a coreset is a weighted subset of the input such that difference between the cost for any set of k centers C on the coreset and the cost on the original point set P is
at most ε · cost ( P, C ) .
Since its initial study by Har-Peled and Mazumdar [2004], the Euclidean k -means problem has received arguably the most attention out of any coreset problem.
The current state of the art by Cohen-Addad et al.
[2022] yields coresets of size ˜ O
( kε − 2 min( k, ε − 2 )) ,
where ˜ O
( x ) hides multiplicative factors that are polylogarithmic in x .
Unfortunately, there is still a gap towards the best known lower bound of Ω( kε − 2 ) by Cohen-Addad et al.
We thus have the option of obtaining either an optimal dependency on k , at the cost of a suboptimal dependency on ε − 1 , or an optimal dependency on ε − 1 , at the cost of a suboptimal dependency on k .
While these bounds suggest that the lower bound is the correct answer, things are not as clear on closer inspection.
Quadratic dependencies on k become necessary for many forms of analysis and so far, it is unknown how to avoid this loss while retaining an optimal dependency on the remaining parameters 1 .
Moreover, the trade-off between the dependency on k and the dependency on ε − 2 is natural.
Specifically, if k ≈ ε − 2 the two previous alternatives ˜ O
( k 2 /ε 2 ) and ˜ O
( k/ε 4 ) are equal.
In this paper we break through this barrier.
Specifically, we show that coresets of size ˜ O
( k 3 / 2 ε − 2 ) exist.
Furthermore, for the Euclidean k -median problem where we minimize cost ( P, C ) := (cid:80)
p ∈
P min c ∈ C ∥
p − c ∥ , we also improve the known bounds from ˜ O
( kε − 2 min( k, ε − 1 )) to ˜ O
( kε − 2 min( k 1 / 3 , ε − 1 )) .
In our view, this is further and arguably stronger evidence that the kε −bound will be the correct answer.
Another contribution exists on a technical level.
Previously, most coreset constructions for high dimensions heavily relied on terminal embeddings to facilitate the analysis.
In this paper, we present a novel method that avoids terminal embeddings.
We expect that our technique may have further applications for coreset constructions in Euclidean spaces.
1.1 Techniques
Starting point of our work is the framework introduced by Cohen-Addad et al.
[2021a] and specifically
Cohen-Addad et al.
Prior to Cohen-Addad et al.
[2022], all coreset analyses required a dependency of at least k · d · ε − 2 .
To illustrate why, suppose we are sampling points from some distribution and wish to use the sampled points as an estimator for the true cost of any candidate solution.
The analysis then consists of (1) a bound on the variance σ 2 of the estimator and (2) a bound on the number of solutions | N | to be approximated.
This typically results in coresets of size
O ( ε − 2 · σ 2 · log | N | ) .
When enumerating all (discretized) candidate solutions (henceforth called a net ) in d dimensions, virtually all known techniques result in | N | ≈ exp( k · d ) .
The dependency on d may be reduced to log( k/ε ) ε − 2 using dimension reduction techniques.
To bypass this, Cohen-Addad et al.
[2022] used a chaining-based analysis to define a sequence of discretized candidate solutions.
Specifically, they showed that there exist discretizations
N α of size exp( kα − 2 ) such that
for any point p and any solution S , there exists a solution S α with
[Equation]
where cost ( p, S α ) =
min s ∈S α ∥ p − s ∥ 2 and cost ( p, S )
=
min s ∈S ∥ p − s ∥ 2 .
The idea of the analysis is then to write cost ( p, S ) as a telescoping sum
[Equation]
and show that the sampled points achieve concentration for each summand.
The number of candidate solutions is now | N 2 − ( h +1) | · | N 2 − h | ≈ exp( k · 2 2 h ) but the difference in cost is 2 − h cost ( p, S ) .
This directly leads to a variance of the order 2 − 2 h σ 2 , where σ 2 is the variance of the basic estimator.
Thus, the increase in net size is countered by the decrease and variance.
Using this technique, Cohen-Addad et al.
[2022] obtained a coreset of size roughly ε − 2 kσ 2 .
To obtain a bound on σ 2 , the framework by Cohen-Addad et al.
[2021a] proposed an algorithm that first computes a solution A with strong structural properties and then samples any point p proportionate cost ( p, A ) .
To simplify the exposition, we assume that A is the optimum, every cluster has identical cost and every point has identical cost.
For this special case, the distribution of CohenAddad et al.
[2021a] turns out to be equivalent to uniform sampling.
For any given solution S , let k i be the number of clusters of A whose points are served at cost 2 i times their cost in A for i > 2 2 , i.e.
[Equation]
p, A ) .
Cohen-Addad et al.
[2021a] showed that their sampling distribution leads to a variance of the order σ 2 ≈ (cid:16)
k · k i 2 i ( k + k i · 2 i ) 2 (cid:17) · min( ε − 2 , 2 i , k ) , which yields the ˜ O ( kε − 2 min( k, ε − 2 )) bound.
To improve either the variance or the net size has to reduced.
Unfortunately, is unlikely that reducing σ 2 will be possible as the bounds on σ 2 obtained by Cohen-Addad et al.
[2022] are tight up to constant factors.
Our main goal is to find a net with an finer error of
[Equation]
In the case of min(2 i , ε − 2 , k ) = 2 i , this leads to a reduced variance for the h -th summand of the order 2 − 2 h · ( √
[Equation]
[Equation] [Equation]
(cid:17) .
To find such a net, we now have two options.
First, we can essentially use the previous nets and rescale 2 − h by a factor 2 − i/ 2 .
Unfortunately, this leads to nets of size exp( k · 2 2 h 2 i ) , so any gain in reducing the variance is countered by an increase in the net size.
The novelty in our approach now lies in showing that a net of size exp( k · k i · 2 2 h ) exists.
Combining the two net sizes with the improved variance bound results in a coreset of size roughly ε − 2 log (cid:0) exp( k · min( k i , 2 i ) · 2 2 h ) (cid:1) · 2 − 2 h · (cid:16) k
[Equation]
k
[Equation]
i ( k + k
[Equation]
2 i ) 2 (cid:17) , which after some calculation is shown to be of the order O ( k 1 .
5 ε − 2 ) .
For the remainder of this section we will illustrate how these improved nets can be obtained.
The key idea already appears in the case of a single center.
Suppose that s is a candidate center.
Then we can show that there always exists a subspace with orthogonal basis U spanned by a set T of at most
O ( α − 2 ) points of P such that
[Equation]
We can then write
[Equation]
( I − UU T ) s.
The error for the terms ∥
Up − s ∥ 2 and ∥ ( I − UU T ) s ∥ 2 can be made negligibly small using a net of size exp( α − 2 ) .
Thus the main loss in error comes from the p T ( I − UU T ) s term.
The key insight is that when adding the center c i of point p in solution A , we have ∥ ( I − UU T ) p ∥ 2 ≤ (cid:112) cost ( p, A ) .
Thus, when adding all the k i centers of clusters that cost 2 i more in S than in A to T the net size becomes exp( α − 2 + k i ) .
By composing these nets for k candidate centers, we then obtain our desired bound.
1.2 Related Work
There has been a tremendous amount of work on coresets for Euclidean k -means following the work in Bachem et al.
[2018a,b], Baker et al.
[2020], Bandyapadhyay et al.
[2021], Becchetti et al.
[2019], Braverman et al.
[2022, 2021b], Chen [2009], Cohen-Addad and Li [2019], CohenAddad et al.
[2021a,b], Feldman and Langberg [2011], Feldman et al.
[2020], Feng et al.
Fichtenberger et al.
[2013], Har-Peled and Kushal [2007], Har-Peled and Mazumdar [2004], Huang and Vishnoi [2020], Huang et al.
[2018, 2019], Langberg and Schulman [2010], Schmidt et al.
Schwiegelshohn and Sheikh-Omar [2022], Sohler and Woodruff [2018].
Almost as prolific is the catalogue of work on dimension reduction for clustering problems in Euclidean spaces, see Boutsidis et al.
[2009, 2010, 2015], Charikar and Waingarten [2022a,b], Cohen et al.
[2015], Cohen-Addad and Schwiegelshohn [2017], Drineas et al.
[2004], Feng et al.
[2019], Makarychev et al.
The arguably most important dimension reduction technique for coresets are terminal embeddings, see Cherapanamjeri and Nelson [2021], Elkin et al.
[2017], Mahabadi et al.
[2018], Narayanan and
Nelson [2019].
Subsequently to our publication, Huang et al.
[2022] obtained similar bounds to our result.
Specifically, they showed a ˜ O ( k (2 z +2) /z
+
ε − 2 ) coreset bound for ( k, z ) clustering, which recovers our result for k -means and k -median and extends it to higher powers.
Further work on coresets considering objects other than points as centers Braverman et al.
[2021a],
Feldman et al.
[2010], Huang et al.
[2021] or other objectives all together Boutsidis et al.
Huang et al.
[2020], Jiang et al.
[2021], Karnin and Liberty [2019], Mai et al.
[2021], Molina et al.
[2018], Munteanu et al.
[2018], Phillips and Tai [2020], Tukan et al.
For further reading, we refer the interested reader to recent surveys Feldman [2020], Munteanu and Schwiegelshohn [2018].
Preliminaries and Setup
First, we require the following basic notions.
For a point p ∈ R d , we denote ∥ p ∥ 2 = (cid:113)(cid:80) di =1 p 2 i to be the Euclidean norm of p and ∥ p ∥ 1 = (cid:80) di
=1 | p i | .
The distinct number of points in a point set P is denoted by ∥ P ∥ 0 .
Note that the true number of points | P | may be larger than ∥ P ∥
as different points may lie on the same coordinates.
Given a solution S consisting of at most k centers, and any subset P ′ ⊂ P we use cost ( P ′ , S ) :
= (cid:80) p ∈ P ′ cost ( p, S )
= (cid:80)
p ∈
P min s ∈S w p cost ( p, s ) , where cost ( p, s ) =
∥ p − s ∥ 2 for Euclidean k -means and w p is a non-negative weight (in the basic case this simply 1 whereas for the coreset it can be any non-negative number).
We also denote by v S ∈ R ∥
P ∥the cost vector associated with the point set P and solution S , that is v S p :=
w p cost ( p, s ) .
Note that ∥ v S ∥ 1 = cost ( P, S ) .
The classic coreset guarantee is to show that for any solution S the designated coreset Ω satisfies
[Equation]
We will later introduce an equivalent statement that uses cost vectors.
It will also be convenient to consider coresets with an additive error E which satisfy
[Equation]
Cohen-Addad et al.
[2022] showed that any coreset algorithm that works for instances with the following assumptions can be extended to general instances:
Assumption 1: ∥ P ∥ 0 ∈ poly ( k, ε − 1 ) .
Assumption 2: d ∈ O (log( k/ε ) · ε − 2 ) .
Assumption 3: w p = 1 , for all p ∈ P .
Note that this only applies to the weights of the original points; the coreset points will have different weights.
Assumption 4: There exists a solution A such that
1. |A| ∈ O ( k ) .
For any two clusters C i , C j induced by A , cost ( C i , A ) ≤ 2 · cost ( C j , A ) .
For any cluster C j induced by A and any two points p, p ′ ∈
C j , cost ( p, A ) ≤ 2 · cost ( p ′ , A )
To keep this paper self contained, we will later detail the validity of these assumptions.
The sampling procedure is now very simple.
Given that these aforementioned assumptions hold, we sample a points p ∈ C j with probability P p := 1 | C j | · cost ( C j , A ) cost ( P, A ) and add it to the designated coreset Ω .
Furthermore, p receives the weight w p = 1 P p .
Overall, our basic cost estimator for any candidate solution S is therefore cost (Ω , S ) :
= 1 | Ω | (cid:80)
p ∈
Ω cost ( p, S ) · w p .
It is routine to check that
E [ cost (Ω , S )]
= cost ( P, S ) .
The remainder of this section will be devoted to showing that Ω satisfies for all S
[Equation]
Using the framework from Cohen-Addad et al.
[2021a], this implies an O ( ε ) coreset in general.
2.1 Justification of the Assumptions
To obtain the first assumption, we compute any coreset of size poly ( k, ε − 1 ) in preprocessing.
Constructions of these coresets are abundant in literature and any one would serve our needs.
To obtain the second assumption, we apply a terminal embedding on the coreset.
A terminal embedding guarantees that for any point p ∈ P and any point q ∈ R d , where d is the dimension of the points of P , we have a mapping f s.t.
[Equation]
Narayanan and Nelson [2019] showed that for any n -point set a terminal embedding of target dimension ˜ O
( ε − 2 log n ) exists, which, combined with the first assumption, yields the desired target dimension.
To obtain the third assumption, we merely have to ensure that the weights of the coreset points are integers.
A number of constructions satisfy this but a simple way of always enforcing this is to scale and round the weights (see Corollary 2 of Cohen-Addad et al.
[2021a]).
The fourth assumption follows from the preprocessing of Cohen-Addad et al.
[2021a], see Sections 3.
and 4.1 of that reference.
Similarly, the same preprocessing, given that A is an O (1) -approximation, also shows that Eq 1 implies that the overall construction will be a coreset (subject to rescaling ε by constant factors), see Section 4.2 of the aforementioned reference.
We must point out that a point set cannot always be decomposed into only sets that satisfy the aforementioned assumption.
Nevertheless
Cohen-Addad et al.
[2022] showed that every other case require only ˜ O ( k/ε 2 ) many sampled points (compared Lemmas 15 and 17 of that reference.)
Finally, we remark that these steps and assumptions immediately also apply to the k -median problem.
Analysis
In this section we prove the following theorem.
Theorem 1.
For any set of points in d dimensional Euclidean space, there exists a coreset for k -means clustering of size ˜ O
( k 1 . 5 ε − 2 ) .
For the k -median case, we also obtain an improved bound as follows.
Details will mainly consider the result for k -means.
The analysis for k -median is given in the supplementary material.
Theorem 2.
For any set of points in d dimensional Euclidean space, there exists a coreset for k -means clustering of size ˜ O
( k 4 / 3 ε − 2 ) .
We first describe the random process used to show concentration of the estimator.
3.1 Setting up the Chaining Analysis
First, we observe that Eq.1 is equivalent to showing
[Equation]
Our goal is to show that
[Equation]
where E Ω is meant to denote the expectation over the randomness of Ω .
This implies that the desired guarantee holds with constant probability.
We now apply a standard symmetrization argument.
Lemma 1 (Appendix B.3 of Rudra and Wootters [2014]) .
Let g p be independent standard Gaussian random variables.
Then.
[Equation]
It is therefore sufficient to show
[Equation]
We partition the clusters of any solution S by type.
We consider a cluster C j of type T i if
[Equation]
The number of clusters C j ∈ T i are denoted by k i .
If C j is of type i ≤
, we say C j is of type T small and if C j is of type i ≥ log γε − 2 , for a sufficiently large absolute constant γ , we say that C j is of type T large .
Then, we show
[Equation]
, S ) w p · g p ( cost ( P, S )
+ cost ( P, A )) (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:35) ≤ ε √ 2 π log 3 ε −(3)
E Ω E g (cid:34) sup S (cid:12) (cid:12) (cid:12)(cid:12)(cid:12) 1 | Ω | (cid:80) C j ∈ T
i (cid:80) p ∈
C
j ∩ Ω cost ( p, S ) w p · g p ( cost ( P, S )
+ cost ( P, A ))
(cid:12) (cid:12) (cid:12)(cid:12)(cid:12)(cid:35) ≤ ε √ 2 π log 3 ε −(4)
[Table]
(cid:34) sup S (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) 1 | Ω | (cid:80) C j ∈ T large (cid:80) p ∈
C
j ∩ Ω cost ( p, S ) w p · g p ( cost ( P, S )
+ cost ( P, A )) (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:35) ≤ ε √ 2 π log 3 ε −(5)
Note that if Equation 4 holds for i ∈ { 3 , . . .
, log 1 /ε } , this also implies Equation 2, as the error from each type can only sum up in the worst case and there are at most O (log ε − 1 ) many types.
The small and large types are comparatively simple to handle.
Lemma 2 (Lemmas 15 and 16 of Cohen-Addad et al.
Let | Ω | ≥ κ kε 2 polylog ( k/ε ) for some absolute constant κ .
Then Equations 3 and 5 hold.
Our main objective will be to prove the following lemma.
Lemma 3.
Let | Ω | ≥ κ · k 1 .
5 ε 2 polylog ( k/ε ) for some absolute constants κ .
Then Equation 4 holds.
Combining Lemma 2 and Lemma 3 then implies Theorem 2.
3.2 Sketch of the Proof of Lemma
The proof of Lemma 3 mainly consists of defining a nested sequence of nets over cost vectors over which we apply a union bound.
Roughly speaking, for any cost vector v S , we aim to find an approximating cost vector v ′ such that
[Equation]
This analysis differs from the terminal-embedding-based nets one used in Cohen-Addad et al.
[2022], which aimed for an error of the order ε · cost ( p, S ) .
Suppose we have, for every ε , a suitable collection of approximating cost vectors N log1 /ε
with this guarantee for any candidate S 3 .
Let v S ,ε be the cost vector approximating v S in the net N log1 /ε .
Then we can write
[Equation]
with v S , 1 p = 0 .
Our goal is to now bound
[Equation]
(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation] [Equation]
We will bound Equations 6 and 8 directly.
For the O (log ε − 1 ) equations in term 7, we prove a bound on each.
Thus, we aim for a bound of the order O ( ε log 3 ε − 1 ) ; the overall bound then follows by summing up the errors and rescaling by constant factors.
Technically, bounding each of the terms in
Equations 6, 7 and 8 requires somewhat different arguments.
For the sake of illustrating the key new ideas we focus on Eq. 7.
A full proof is contained in the supplementary material.
The next section presents the nets for the cost vectors.
The subsequent section bounds the variance.
The final section combines these results and completes the proof of Lemma 3.
Cost Vector Nets
Definition 1.
Let I be a metric space, P a set of points, k a positive integer, and let α > 0 be a precision parameters and let A be some solution with at most k ′ centers.
Let C ⊂
I k be a (potentially infinite) set of candidate k -clusterings.
We say that a set of cost vectors N ⊂ R | P | is an ( α, k ) -means clustering net if for every S ∈ C there exists a vector v ′ ∈ N such that the following condition holds.
For all p ∈ P ,
[Equation]
These nets have a substantially smaller error than those proposed in Cohen-Addad et al.
[2022], which had an error of the order α · ( cost ( p, S )
+ cost ( p, A )) .
Given a set of points X in Euclidean space, an ε -net is a subset S ⊂ X such that for every p ∈
X there exists a q in S with ∥ p − q ∥ ≤ ε .
Throughout this section, we will frequently use the fact that in d dimensions, there exists an ε -net of cardinality (1 + 2 /ε )
d (see for example Pisier [1999]).
Our main goal in this section is to prove the following lemma.
Lemma 4.
Let P be a set of points in d dimensional Euclidean space, k a positive integer, A be a candidate solution with k i clusters and γ an absolute constant.
Define C to be the set of possible candidate centers such that a subset A i the clusters induced by A are of type i , with 3 ≤ i ≤ log 1 /ε 2 .
For all α ≤ 1 / 2 , there exists an ( α, k ) -means clustering net N of C with
[Equation]
Proof (Sketch).
We first show that given a set of vectors P and any vector s , there always exists a small subset U of P such that all inner products between p ∈ P and s are preserved by the span of U .
Lemma 5.
Let P = { p 1 , ..., p n }
⊆
R
d and let s ∈ R d .
Then there exists U ⊆ P , with | U |
=
O ( ε − 2 ) and orthogonal basis Π U , such that
[Equation]
Proof.
Start with U 0 = argmin p ∈ P ∥
p − s ∥ , and proceed in rounds.
Note that ∥ ( I − Π U
Π TU 0 ) s ∥ ≤ ∥ p − s ∥ for all p ∈ P .
In each round i , denote the current set of vectors
U i with orthogonal basis Π U i .
We add a vector p i if the following equation holds
[Equation]
We observe that if this equation holds for all p ∈ P , then Equation 9 must also hold.
Note that ( I − Π U
i Π TU i ) p is orthogonal to the span of U i of all previously added vectors.
Thus, due to the
Pythagorean theorem, we have
[Equation]
Therefore, after t = ε − 2 many rounds ( I − Π U Π TU ) s = 0 , which implies that after at most ε −rounds Eq.
9 has to hold.
With this lemma, we can prove our net bound.
Our objective is to generate a small set of cost vectors that satisfy the desired guarantee.
We first define the cost vectors.
For each subset U of size O (min( α − 2
i , α − 2 + k i ) , we consider the the subspace
Π U spanned by U .
In this subspace we consider ( α/ 2 i ) · (cid:112) cost ( p, A ) -nets of every ball centered around Π U p with radius 60 · 2 i / 2 · (cid:112) cost ( p, A ) for all p ∈ P .
Such a net has size exp( γ · rank ( U ) i log α ) , for some constant γ and there exist at most
∥ P ∥ 0 · exp( γ · | U | i log α )
many such nets.
Furthermore, there are at most (cid:0)
∥
P ∥ 0 | U | (cid:1) ≤
∥ P ∥ | U | 0 such subsets.
Now, for every point p , define an exponential sequence α 2 (1 + α/ 2 i ) j for j ∈ { 0 , . .
. log 10 · 2 i } .
There exist at most ∥ P ∥ 0 such sequences and every such sequence consists of at most O ( α − 1 · 2 i · i ) many values.
We combine every net point in ever ball of every subspace with all values in the exponential sequence to obtain the evaluation for a single candidate center.
The overall number of candidate centers is therefore of the order ∥ P ∥
| U | 0 · exp( γ · | U | i log α ) , for a sufficiently large γ .
The overall number of candidate cost vectors is now the number of k subsets of candidate centers, i.e.
∥ P ∥ k ·| U |· exp( γ · k · | U | i log α ) .
Combined with the bounds on U , this yields the desired size.
What remains to be shown is that the thus constructed cost vectors are a ( α, k ) -clustering net.
Here, we use that for any center s in some candidate solution S
[Equation]
The nets for the span of Π U are so fine that the distance ∥
Π U s − s ′
∥ 2 is essentially negligible, where s ′ is the point in the span of Π U closest to Π U s and the same holds for the exponential sequence approximating the term ∥ ( I − Π U Π TU ) s ∥ 2 .
Thus, the error is dominated by 2 p T ( I − Π U Π TU ) s .
Now, we can assume that the input point closest to s is included in U .
Then min p ∈ P ∥
p − s ∥ ≤ (cid:112) cost ( p, S ) ≤ O (1) · 2 i/ 2 (cid:112) cost ( p, A ) and ∥
( I − Π U Π TU ) p ∥ ≤ (cid:112) cost ( p, S ) ≤ O (1) · 2 i/ 2 (cid:112) cost ( p, A ) .
If α − 2 · 2 i < k i + α − 2 , we have
[Equation]
cost ( p, S ) cost ( p, A ) otherwise we have ∥ ( I − Π U Π TU ) p ∥
≤ (cid:112) cost ( p, A ) which implies
[Equation]
Rescaling α by constant factors yields the claim.
Corollary 3.
Let P be a set of points in d dimensional Euclidean space, k a positive integer, A be a candidate solution with O ( k ) clusters and γ an absolute constant.
Then there exists an ( α, k ) clustering net of size Define C to be the set of possible candidate centers.
such that a subset A i the clusters induced by A are of type i , with 3 ≤ i ≤ log 1 /ε 2 .
For all α ≤ 1 / 2 , there exists an ( α, k ) -means clustering net N of C with
[Equation]
Bounding the Variance
We now use the cost vectors to obtain an improved variance for the estimator
[Equation]
The bounds on variance for any random variable (cid:80) a p g p with standard Gaussians g p is Gaussian distributed with mean 0 and variance (cid:80) a 2 p .
Before we do this, we require an additional notion.
Let E denote the event that 1 | Ω | (cid:80)
p ∈ C
j ∩ Ω w p
=
(1 ± ε ) · | C j | .
The following lemma bounds the probability of E occurring.
Lemma 6.
[Compare Lemma 19 of Cohen-Addad et al.
If Assumption 4 holds, then event E holds with probability 1 − k − 2 if | Ω | > κ · kε − 2 log k for a sufficiently high absolute constant κ .
Lemma 7.
Given Assumption 4, the variance of (cid:80) Cj ∈
Ti (cid:80) p
∈
Cj ∩ Ω ( v S , 2 − ( h +1)
p − v S , 2 − h p ) w p · g p | Ω |· ( cost ( P, S )
+ cost ( P, A )) is at most γ ·
[Equation]
− 2 h |
[Equation]
|
[Equation]
i ·
[Equation]
i ( k
[Equation]
i
[Equation]
i )conditioned on event E
[Equation]
conditioned on event E for an absolute constant γ .
Proof.
We first observe that since the g p are standard normal Gaussians, the entire estimator is
Gaussian distributed with variance
[Equation]
We have ( v S , 2 − ( h +1) p − v S , 2 − h p ) = ( v S , 2 − ( h +1)
p − cost ( p, S )
+ cost ( p, S )
− v S , 2 − h p ) ≤ 2 · 2 − h · (cid:112) cost ( p, A ) cost ( p, S ) due to Lemma 4.
Furthermore, by definition w p
=
cost ( P, A )
| C j | cost ( C j , A ) .
Finally, by definition of type i , we have cost ( p, S ) · | C j | =
O (1) · cost ( C j , S ) and by Assumption 4 we have cost ( p, A ) · | C j | = O (1) · cost ( C j , A ) for all p ∈ C j . (cid:88) C j ∈ T i (cid:88) p ∈ C
j ∩ Ω | Ω |(cid:32) (
[Equation]
S
[Equation]
( h
[Equation]
1) p − v S [Equation] 2 − h p ) w p ( cost ( P, S ) + cost ( P, A ))
(cid:33)≤ O (1) · (cid:88)
C j
[Equation]
T i (cid:88) p
[Equation]
C
[Equation]
Ω | Ω |(cid:32)
[Equation]
h · (cid:112) cost ( p, A ) cost ( p, S ) · cost ( P, A ) | C j | cost ( C j , A ) · ( cost ( P, S )
+ cost ( P, A ))
(cid:33)≤ O (1) · (cid:88)
C j
[Equation]
T i (cid:88) p
[Equation]
C j ∩ Ω | Ω |(cid:32) 2 − 2 h · cost ( C j , A ) · cost ( C j , S ) cost ( P, A )cost ( C j , A ) 2 · ( cost ( P, S )
+ cost ( P, A ))(cid:33)
Now, let k i be the number of clusters of type i .
Then due to Assumption 4 cost ( C j , S ) · k i ≤
O (1) cost ( P, S ) , for all C j of type i .
Finally, note that cost ( P, A ) cost ( C j , A ) ≤
O (1) · k , also due to Assumption 4.
Combining this, we then have
[Equation]
Assuming event E , this may now be bounded by O (1) · 2 − 2 h | Ω | · k · k i · 2 i [Equation] k + k i · 2 i
[Equation]
If event E does not hold, we may bound the term by 2 −
[Equation]
Completing the Proof (for Eq. 7)
Throughout this section, we use the bound on the expected maximum of independent Gaussians.
Lemma 8 (Lemma 2.3 of Massart [2007]) .
Let g i ∼ N (0 , σ 2 i ) , i ∈ [ n ] be Gaussian random variables and suppose σ i ≤ σ for all i .
Then E [max i ∈ [ n ] | g i | ] ≤ 2 σ · √ 2 ln n.
We construct nets for all cost vectors constrained to the entries associated with points of clusters of type i in some solution, i.e. we only consider points p ∈ C j ∈ T i and the entries v S p .
For every fixed set of clusters of size k i of type T i , the number of cost vectors in N h
+1 × N h is at most exp (cid:0) γ · k · log ∥
P ∥ 0 · min( k i + 2 2 h , 2 2 h · 2 i ) · i · h ) (cid:1) for some absolute constant γ due to
Lemma 4.
We split the vectors in the nets by k i ≤ √ k or by k i ≥ √ k . In both cases, we have min( k i , 2 i ) k
· k i · 2 i ( k + k i · 2 i ) 2 ≤ √ k .
With the bound on the variance (Lemma 7 and conditioned on event E ), we then have
[Equation]
(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) E  
[Table] 

[Table]
[Equation] If E does not hold, we use the somewhat coarse bound of min( k i , 2 i ) k · k t · 2 i [Equation]
k + k t · 2 i ) 2 ≤ k to obtain log
[Equation]
(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)
[Equation]
(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) E   ≤
[Equation]
We have P [ E ] ≤ 1 /k 2 due to Lemma 6.
Since ∥ P ∥ 0 ≤ poly ( k, ε − 1 ) , 2 i ≤ O (1) · ε − 2 and by our choice of | Ω | , we can combine Equations 10 and 11 with the law of total expectation to obtain log [Equation] (cid:88) [Equation]
(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)
[Equation]
Disclosure of Funding Acknowledgements
Kapser Green Larsen was partially supported by the Independent Research Fund Denmark (DFF) under a Sapere Aude Research Leader grant No 9064-00068B.
David Saulpic has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (Grant agreement No.
“The Design of Modern Fully Dynamic Data
Structures (MoDynStruct)”.
Chris Schwiegelshohn was partially supported by the Independent Research Fund Denmark (DFF) under a Sapere Aude Research Leader grant No 1051-00106B and the Innovation Fund Denmark under grant agreement No 0153-00233A. Omar Ali Sheikh-Omar was partially supported by the Innovation Fund Denmark under grant agreement No 0153-00233A.