Improved Coresets for Euclidean k -Means
Abstract
Given a set of n points in d dimensions, the Euclidean k -means problem consists of finding k centers such that the sum of squared distances from every point to its closest center is minimized.
The arguably most popular way of dealing with this problem in the big data setting is to first compress the data by computing a weighted subset known as a coreset and then run any algorithm on this subset.
The guarantee of the coreset is that for any candidate solution, the ratio between coreset cost and the cost of the original instance is less than a (1 ± ε ) factor.
The current state of the art coreset size for Euclidean k -means is ˜ O
(min( k 2 · ε − 2 , k · ε − 4 )) .
This matches the lower bound of Ω( kε − 2 ) up to a min( k, ε − 2 ) factor.
In this paper, we improve this bound to ˜ O (min( k 1 . 5 · ε − 2 , k · ε − 4 )) .
In the regime where k ≤ ε − 2 , this is a strict improvement over the state of the art.
In particular, ours is the first provable bound that breaks through the k 2 barrier while retaining an optimal dependency on ε .
Introduction
Coresets have become a staple in the design of algorithms for large data sets.
In the most general setting, a coreset compresses the data set in such a way that for any set of previously specified candidate queries, the cost of evaluating the query and the cost of the coreset are similar, up to an arbitrary small distortion.
A popular subject in coreset literature is the Euclidean k -means problem.
Here, we are given n points P in d dimensions and our task is to find a set of k points C called centers minimizing cost ( P, C ) :
= (cid:80)
p ∈
P min c ∈ C ∥
p − c ∥ 2 , where ∥ p − c ∥ =
(cid:113)(cid:80) di
=1 ( p i − c i ) 2 denotes the Euclidean distance.
In this case, a coreset is a weighted subset of the input such that difference between the cost for any set of k centers C on the coreset and the cost on the original point set P is
at most ε · cost ( P, C ) .
Since its initial study by Har-Peled and Mazumdar [2004], the Euclidean k -means problem has received arguably the most attention out of any coreset problem.
The current state of the art by Cohen-Addad et al.
[2022] yields coresets of size ˜ O
( kε − 2 min( k, ε − 2 )) ,
where ˜ O
( x ) hides multiplicative factors that are polylogarithmic in x .
Unfortunately, there is still a gap towards the best known lower bound of Ω( kε − 2 ) by Cohen-Addad et al.
We thus have the option of obtaining either an optimal dependency on k , at the cost of a suboptimal dependency on ε − 1 , or an optimal dependency on ε − 1 , at the cost of a suboptimal dependency on k .
While these bounds suggest that the lower bound is the correct answer, things are not as clear on closer inspection.
Quadratic dependencies on k become necessary for many forms of analysis and so far, it is unknown how to avoid this loss while retaining an optimal dependency on the remaining parameters 1 .
Moreover, the trade-off between the dependency on k and the dependency on ε − 2 is natural.
Specifically, if k ≈ ε − 2 the two previous alternatives ˜ O
( k 2 /ε 2 ) and ˜ O
( k/ε 4 ) are equal.
In this paper we break through this barrier.
Specifically, we show that coresets of size ˜ O
k 1 .
ε − 2 ) exist.
In our view, this is further and arguably stronger evidence that the kε − 2 bound will be the correct answer.
Another contribution exists on a technical level.
Previously, most coreset constructions for high dimensions heavily relied on terminal embeddings to facilitate the analysis.
In this paper, we present a novel method that avoids terminal embeddings.
We expect that our technique may have further applications for coreset constructions in Euclidean spaces.
1.1 Techniques
Starting point of our work is the framework introduced by Cohen-Addad et al.
[2021a] and specifically
Cohen-Addad et al.
Prior to Cohen-Addad et al.
[2022], all coreset analyses required a dependency of at least k · d · ε − 2 .
To illustrate why, suppose we are sampling points from some distribution and wish to use the sampled points as an estimator for the true cost of any candidate solution.
The analysis then consists of (1) a bound on the variance σ 2 of the estimator and (2) a bound on the number of solutions | N | to be approximated.
This typically results in coresets of size
O ( ε − 2 · σ 2 · log | N | ) .
When enumerating all (discretized) candidate solutions (henceforth called a net ) in d dimensions, virtually all known techniques result in | N | ≈ exp( k · d ) .
The dependency on d may be reduced to log( k/ε ) ε − 2 using dimension reduction techniques.
To bypass this, Cohen-Addad et al.
[2022] used a chaining-based analysis to define a sequence of discretized candidate solutions.
Specifically, they showed that there exist discretizations
N α of size exp( kα − 2 ) such that
for any point p and any solution S , there exists a solution S α with | cost ( p, S α ) − cost ( p
[Equation]
· cost ( p, S ) , where cost ( p, S α ) =
min s ∈S α ∥ p − s ∥ 2 and cost ( p, S )
=
min s ∈S ∥ p − s ∥ 2 .
The idea of the analysis is then to write cost ( p, S ) as a telescoping sum
[Equation]
and show that the sampled points achieve concentration for each summand.
The number of candidate solutions is now | N 2 − ( h +1) | · | N 2 − h | ≈ exp( k · 2 2 h ) but the difference in cost is 2 − h cost ( p, S ) .
This directly leads to a variance of the order 2 − 2 h σ 2 , where σ 2 is the variance of the basic estimator.
Thus, the increase in net size is countered by the decrease and variance.
Using this technique, Cohen-Addad et al.
[2022] obtained a coreset of size roughly ε − 2 kσ 2 .
To obtain a bound on σ 2 , the framework by Cohen-Addad et al.
[2021a] proposed an algorithm that first computes a solution A with strong structural properties and then samples any point p proportionate cost ( p, A ) .
To simplify the exposition, we assume that A is the optimum, every cluster has identical cost and every point has identical cost.
For this special case, the distribution of CohenAddad et al.
[2021a] turns out to be equivalent to uniform sampling.
For any given solution S , let k i be the number of clusters of A whose points are served at cost 2 i times their cost in A for i > 2 2 , i.e.
2 i = cost ( p, S ) cost ( p, A ) .
Cohen-Addad et al.
[2021a] showed that their sampling distribution leads to a variance of the order σ 2 ≈ (cid:16)
k · k i 2 i ( k + k i · 2 i ) 2 (cid:17) · min( ε − 2 , 2 i , k ) , which yields the ˜ O ( kε − 2 min( k, ε − 2 )) bound.
To improve either the variance or the net size has to reduced.
Unfortunately, is unlikely that reducing σ 2 will be possible as the bounds on σ 2 obtained by Cohen-Addad et al.
[2022] are tight up to constant factors.
Our main goal is to find a net with an finer error of
[Equation] In the case of min(2 i , ε − 2 , k ) = 2 i , this leads to a reduced variance for the h -th summand of the order 2 − 2 h · ( [Equation] cost
[Equation]
To find such a net, we now have two options.
First, we can essentially use the previous nets and rescale 2 − h by a factor 2 − i/ 2 .
Unfortunately, this leads to nets of size exp( k · 2 2 h 2 i ) , so any gain in reducing the variance is countered by an increase in the net size.
The novelty in our approach now lies in showing that a net of size exp( k · k i · 2 2 h ) exists.
Combining the two net sizes with the improved variance bound results in a coreset of size roughly ε − 2 log (cid:0) exp( k · min( k i , 2 i ) · 2 2 h ) (cid:1) · 2 − 2 h · (cid:16) k · k i · 2 i ( k + k i · 2 i ) 2 (cid:17) , which after some calculation is shown to be of the order O ( k 1 .
5 ε − 2 ) .
For the remainder of this section we will illustrate how these improved nets can be obtained.
The key idea already appears in the case of a single center.
Suppose that s is a candidate center.
Then we can show that there always exists a subspace with orthogonal basis U spanned by a set T of at most
O ( α − 2 ) points of P such that
[Equation]
We can then write
[Equation]
The error for the terms ∥
Up − s ∥ 2 and ∥ ( I − UU T ) s ∥ 2 can be made negligibly small using a net of size exp( α − 2 ) .
Thus the main loss in error comes from the p T ( I − UU T ) s term.
The key insight is that when adding the center c i of point p in solution A , we have ∥ ( I − UU T ) p ∥ 2 ≤ (cid:112) cost ( p, A ) .
Thus, when adding all the k i centers of clusters that cost 2 i more in S than in A to T the net size becomes exp( α − 2 + k i ) .
By composing these nets for k candidate centers, we then obtain our desired bound.
1.2 Related Work
There has been a tremendous amount of work on coresets for Euclidean k -means following the work in Bachem et al.
[2018a,b], Baker et al.
[2020], Bandyapadhyay et al.
[2021], Becchetti et al.
[2019], Braverman et al.
[2022, 2021b], Chen [2009], Cohen-Addad and Li [2019], CohenAddad et al.
[2021a,b], Feldman and Langberg [2011], Feldman et al.
[2020], Feng et al.
Fichtenberger et al.
[2013], Har-Peled and Kushal [2007], Har-Peled and Mazumdar [2004], Huang and Vishnoi [2020], Huang et al.
[2018, 2019], Langberg and Schulman [2010], Schmidt et al.
Schwiegelshohn and Sheikh-Omar [2022], Sohler and Woodruff [2018].
Almost as prolific is the catalogue of work on dimension reduction for clustering problems in Euclidean spaces, see Boutsidis et al.
[2009, 2010, 2015], Charikar and Waingarten [2022a,b], Cohen et al.
[2015], Cohen-Addad and Schwiegelshohn [2017], Drineas et al.
[2004], Feng et al.
[2019], Makarychev et al.
The arguably most important dimension reduction technique for coresets are terminal embeddings, see Cherapanamjeri and Nelson [2021], Elkin et al.
[2017], Mahabadi et al.
[2018], Narayanan and
Nelson [2019].
Further work on coresets considering objects other than points as centers Braverman et al.
[2021a],
Feldman et al.
[2010], Huang et al.
[2021] or other objectives all together Boutsidis et al.
Huang et al.
[2020], Jiang et al.
[2021], Karnin and Liberty [2019], Mai et al.
[2021], Molina et al.
[2018], Munteanu et al.
[2018], Phillips and Tai [2020], Tukan et al.
For further reading, we refer the interested reader to recent surveys Feldman [2020], Munteanu and Schwiegelshohn [2018].
1.3 Limitations and Societal Impact
The results do not extend to higher powers, i.e. for cost ( p, s ) =
∥ p − s ∥
z the bound ˜ O
( kε − 2 · min( k, ε − z )) remains the state of the art.
For k -median the new bound also holds (discussed in the supplementary material).
Arguably, the ˜ O ( k 1 . 5 ε − 2 ) bound also suggests that a ˜ O ( k · ε − 3 ) bound should be possible for k -means.
We were not able to obtain this and ˜ O
( kε − 4 ) remains the state of the art if the dependency on k is required to be (nearly) linear.
Since this is mainly a theoretical work on a basic computational problem, we do not believe that there is any adverse societal impact.
Preliminaries and Setup
First, we require the following basic notions.
For a point p ∈ R d , we denote ∥ p ∥ 2 = (cid:113)(cid:80) di =1 p 2 i to be the Euclidean norm of p and ∥ p ∥ 1 = (cid:80) di
=1 | p i | .
The distinct number of points in a point set P is denoted by ∥ P ∥ 0 .
Note that the true number of points | P | may be larger than ∥ P ∥
as different points may lie on the same coordinates.
Given a solution S consisting of at most k centers, and any subset P ′ ⊂ P we use cost ( P ′ , S ) :
= (cid:80) p ∈ P ′ cost ( p, S )
= (cid:80)
p ∈
P min s ∈S w p cost ( p, s ) , where cost ( p, s ) =
∥ p − s ∥ 2 for Euclidean k -means and w p is a non-negative weight (in the basic case this simply 1 whereas for the coreset it can be any non-negative number).
We also denote by v S ∈ R ∥
P ∥the cost vector associated with the point set P and solution S , that is v S p :=
w p cost ( p, s ) .
Note that ∥ v S ∥ 1 = cost ( P, S ) .
The classic coreset guarantee is to show that for any solution S the designated coreset Ω satisfies |
[Equation]
We will later introduce an equivalent statement that uses cost vectors.
It will also be convenient to consider coresets with an additive error E which satisfy
[Equation]
Cohen-Addad et al.
[2022] showed that any coreset algorithm that works for instances with the following assumptions can be extended to general instances:
Assumption 1: ∥ P ∥ 0 ∈ poly ( k, ε − 1 ) .
Assumption 2: d ∈ O (log( k/ε ) · ε − 2 ) .
Assumption 3: w p = 1 , for all p ∈ P .
Note that this only applies to the weights of the original points; the coreset points will have different weights.
Assumption 4: There exists a solution A such that
1. |A| ∈ O ( k ) .
For any two clusters C i , C j induced by A , cost ( C i , A ) ≤ 2 · cost ( C j , A ) .
For any cluster C j induced by A and any two points p, p ′ ∈ C j , cost [Equation]
p [Equation] A [Equation]
To keep this paper self contained, we will detail the validity of these assumptions at the end of this section.
The sampling procedure is now very simple.
Given that these aforementioned assumptions hold, we sample a points p ∈ C j with probability P p := 1 | C j | · cost ( C j , A ) cost ( P, A ) and add it to the designated coreset Ω .
Furthermore, p receives the weight w p = 1 P p .
Overall, our basic cost estimator for any candidate solution S is therefore
[Equation]
It is routine to check that E [ cost (Ω , S )]
= cost ( P, S ) .
The remainder of this section will be devoted to showing that Ω satisfies for all S
[Equation]
Using the framework from Cohen-Addad et al.
[2021a], this implies an O ( ε ) coreset in general.
2.1 Justification of the Assumptions
To obtain the first assumption, we compute any coreset of size poly ( k, ε − 1 ) in preprocessing.
Constructions of these coresets are abundant in literature and any one would serve our needs.
To obtain the second assumption, we apply a terminal embedding on the coreset.
A terminal embedding guarantees that for any point p ∈ P and any point q ∈ R d , where d is the dimension of the points of P , we have a mapping f s.t.
[Equation]
Narayanan and Nelson [2019] showed that for any n -point set a terminal embedding of target dimension ˜ O
( ε − 2 log n ) exists, which, combined with the first assumption, yields the desired target dimension.
To obtain the third assumption, we merely have to ensure that the weights of the coreset points are integers.
A number of constructions satisfy this but a simple way of always enforcing this is to scale and round the weights (see Corollary 2 of Cohen-Addad et al.
[2021a]).
The fourth assumption follows from the preprocessing of Cohen-Addad et al.
[2021a], see Sections 3.
and 4.1 of that reference.
Similarly, the same preprocessing, given that A is an O (1) -approximation, also shows that Eq 1 implies that the overall construction will be a coreset (subject to rescaling ε by constant factors), see Section 4.2 of the aforementioned reference.
We must point out that a point set cannot always be decomposed into only sets that satisfy the aforementioned assumption.
Nevertheless
Cohen-Addad et al.
[2022] showed that every other case require only ˜ O ( k/ε 2 ) many sampled points (compared Lemmas 15 and 17 of that reference.)
Finally, we remark that these steps and assumptions immediately also apply to the k -median problem.
Analysis
In this section we prove the following theorem.
Theorem 1.
For any set of points in d dimensional Euclidean space, there exists a coreset for k -means clustering of size ˜ O
( k 1 . 5 ε − 2 ) .
We first describe the random process used to show concentration of the estimator.
3.1 Setting up the Chaining Analysis
First, we observe that Eq.1 is equivalent to showing
[Equation]
Our goal is to show that
[Equation]
where E Ω is meant to denote the expectation over the randomness of Ω .
This implies that the desired guarantee holds with constant probability.
We now apply a standard symmetrization argument.
Lemma 1 (Appendix B.3 of Rudra and Wootters [2014]) .
Let g p be independent standard Gaussian random variables.
Then.
[Equation]
It is therefore sufficient to show
[Equation]
We partition the clusters of any solution S by type.
We consider a cluster C j of type T i if
[Equation]
i
[Equation]
min c ∈A ∥ p − c ∥
[Equation]
The number of clusters C j ∈ T i are denoted by k i .
If C j is of type i ≤
, we say C j is of type T small and if C j is of type i ≥ log γε − 2 , for a sufficiently large absolute constant γ , we say that C j is of type T large .
Then, we show
[Equation]
Note that if Equation 4 holds for i ∈ { 3 , . . .
, log 1 /ε } , this also implies Equation 2, as the error from each type can only sum up in the worst case and there are at most O (log ε − 1 ) many types.
The small and large types are comparatively simple to handle.
Lemma 2 (Lemmas 15 and 16 of Cohen-Addad et al.
Let | Ω | ≥ κ kε 2 log 10 ( k/ε ) for some absolute constant κ .
Then Equations 3 and 5 hold.
Our main objective will be to prove the following lemma.
[Equation]
Let | Ω
[Equation]
≥
[Equation]
for some absolute constants κ 1 and κ 2 .
Then Equation 4 holds.
Combining Lemma 2 and Lemma 3 then implies Theorem 1.
3.2 Sketch of the Proof of Lemma
The proof of Lemma 3 mainly consists of defining a nested sequence of nets over cost vectors over which we apply a union bound.
Roughly speaking, for any cost vector v S , we aim to find an approximating cost vector v ′ such that
[Equation]
This analysis differs from the terminal-embedding-based nets one used in Cohen-Addad et al.
[2022], which aimed for an error of the order ε · cost ( p, S ) .
Suppose we have, for every ε , a suitable collection of approximating cost vectors N log1 /ε
with this guarantee for any candidate S 3 .
Let v S ,ε be the cost vector approximating v S in the net N log1 /ε .
Then we can write
[Equation]
with v S , 1 p = 0 .
Our goal is to now bound
[Equation]
[Table]
[Equation]
[Equation]
We will bound Equations 6 and 8 directly.
For the O (log ε − 1 ) equations in term 7, we prove a bound on each.
Thus, we aim for a bound of the order O ( ε log 3 ε − 1 ) ; the overall bound then follows by summing up the errors and rescaling by constant factors.
Technically, bounding each of the terms in Equations 6, 7 and 8 requires somewhat different arguments.
For the sake of illustrating the key new ideas we focus on Eq. 7.
A full proof is contained in the supplementary material.
The next section presents the nets for the cost vectors.
The subsequent section bounds the variance.
The final section combines these results and completes the proof of Lemma 3.
Cost Vector Nets
Definition 1.
Let I be a metric space, P a set of points, k a positive integer, and let α > 0 be a precision parameters and let A be some solution with at most k ′ centers.
Let C ⊂
I k be a (potentially infinite) set of candidate k -clusterings.
We say that a set of cost vectors N ⊂ R | P | is an ( α, k ) -clustering net if for every S ∈ C there exists a vector v ′ ∈ N such that the following condition holds.
For all p ∈ P ,
[Equation]
These clustering nets have a substantially smaller error than those proposed in Cohen-Addad et al.
[2022], which had an error of the order α · ( cost ( p, S )
+ cost ( p, A )) .
Given a set of points X in Euclidean space, an ε -net is a subset S ⊂ X such that for every p ∈
X there exists a q in S with ∥ p − q ∥ ≤ ε .
Throughout this section, we will frequently use the fact that in d dimensions, there exists an ε -net of cardinality (1 + 2 /ε )
d (see for example Pisier [1999]).
Our main goal in this section is to prove the following lemma.
Lemma 4.
Let P be a set of points in d dimensional Euclidean space, k a positive integer, A be a candidate solution with k i clusters and γ and absolute constant.
Define C to be the set of possible candidate centers such that the clusters induced by A are of type i , with 3 ≤ i ≤ log 1 /ε 2 .
For all α ≤ 1 / 2 , there exists an ( α, k ) -clustering net N of C with
[Equation]
Proof (Sketch).
We first show that given a set of vectors P and any vector s , there always exists a small subset U of P such that all inner products between p ∈ P and s are preserved by the span of U .
Lemma 5.
Let P = { p 1 , ..., p n }
⊆
R
d and let s ∈ R d .
Then there exists U ⊆ P , with | U |
=
O ( ε − 2 ) and orthogonal basis Π U , such that
[Equation]
Proof.
Start with U 0 = argmin p ∈ P ∥
p − s ∥ , and proceed in rounds.
Note that ∥ ( I − Π U
Π TU 0 ) s ∥ ≤ ∥ p − s ∥ for all p ∈ P .
In each round i , denote the current set of vectors
U i with orthogonal basis Π U i .
We add a vector p i if the following equation holds
[Equation]
We observe that if this equation holds for all p ∈ P , then Equation 9 must also hold.
Note that ( I − Π U
i Π TU i ) p is orthogonal to the span of U i of all previously added vectors.
Thus, due to the
Pythagorean theorem, we have
[Equation]
Therefore, after t = ε − 2 many rounds ( I − Π U Π TU ) s = 0 , which implies that after at most ε −rounds Eq.
9 has to hold.
With this lemma, we can prove our net bound.
Our objective is to generate a small set of cost vectors that satisfy the desired guarantee.
We first define the cost vectors.
For each subset U of size O (min( α − 2
i , α − 2 + k i ) , we consider the the subspace
Π U spanned by U .
In this subspace we consider ( α/ 2 i ) · (cid:112) cost ( p, A ) -nets of every ball centered around Π U p with radius 60 · 2 i / 2 · (cid:112) cost ( p, A ) for all p ∈ P .
Such a net has size exp( γ · rank ( U ) i log α ) , for some constant γ and there exist at most
∥ P ∥ 0 · exp( γ · | U | i log α )
many such nets.
Furthermore, there are at most (cid:0)
∥
P ∥ 0 | U | (cid:1) ≤
∥ P ∥ | U | 0 such subsets.
Now, for every point p , define an exponential sequence α 2 (1 + α/ 2 i ) j for j ∈ { 0 , . .
. log 10 · 2 i } .
There exist at most ∥ P ∥ 0 such sequences and every such sequence consists of at most O ( α − 1 · 2 i · i ) many values.
We combine every net point in ever ball of every subspace with all values in the exponential sequence to obtain the evaluation for a single candidate center.
The overall number of candidate centers is therefore of the order ∥ P ∥
| U | 0 · exp( γ · | U | i log α ) , for a sufficiently large γ .
The overall number of candidate cost vectors is now the number of k subsets of candidate centers, i.e.
∥ P ∥ k ·| U |· exp( γ · k · | U | i log α ) .
Combined with the bounds on U , this yields the desired size.
What remains to be shown is that the thus constructed cost vectors are a ( α, k ) -clustering net.
Here, we use that for any center s in some candidate solution S
[Equation]
The nets for the span of Π U are so fine that the distance ∥
Π U s − s ′
∥ 2 is essentially negligible, where s ′ is the point in the span of Π U closest to Π U s and the same holds for the exponential sequence approximating the term ∥ ( I − Π U Π TU ) s ∥ 2 .
Thus, the error is dominated by 2 p T ( I − Π U Π TU ) s .
Now, we can assume that the input point closest to s is included in U .
Then min p ∈ P ∥
p − s ∥ ≤ (cid:112) cost ( p, S ) ≤ O (1) · 2 i/ 2 (cid:112) cost ( p, A ) and ∥
( I − Π U Π TU ) p ∥ ≤ (cid:112) cost ( p, S ) ≤ O (1) · 2 i/ 2 (cid:112) cost ( p, A ) .
If α − 2 · 2 i < k i + α − 2 , we have | p T ( I − Π U Π TU ) s | ≤ α · 2 − i/ 2 · ∥ ( I − Π U Π TU ) p ∥ · min p ∈ P ∥ p − s ∥
≤ O (1) · α · (cid:112) cost ( p, S ) cost ( p, A ) otherwise we have ∥ ( I − Π U Π TU ) p ∥ ≤ (cid:112) cost ( p, A ) which implies | p T ( I − Π U Π TU ) s | ≤ α · ∥ ( I − Π U Π TU ) p ∥ · min p ∈ P ∥ p − s ∥ ≤ α · (cid:112) cost ( p, S ) cost ( p, A ) .
Rescaling α by constant factors yields the claim.
Bounding the Variance
We now use the cost vectors to obtain an improved variance for the estimator
[Equation]
g p .
The bounds on variance for any random variable (cid:80)
a
[Equation]
g p with standard Gaussians g p is Gaussian distributed with mean 0 and variance (cid:80) a 2 p .
Before we do this, we require an additional notion.
Let E denote the event that 1 | Ω | (cid:80)
p ∈ C
j ∩ Ω w p
=
(1 ± ε ) · | C j | .
The following lemma bounds the probability of E occurring.
Lemma 6.
[Compare Lemma 19 of Cohen-Addad et al.
If Assumption 4 holds, then event E holds with probability 1 − k − 2 if | Ω | > κ · kε − 2 log k for a sufficiently high absolute constant κ .
Lemma 7.
Given Assumption 4, the variance of (cid:80) Cj ∈
Ti (cid:80) p
∈
Cj
[Equation]
S
[Equation]
, A )) is at most γ · 2 − 2 h | Ω | · k · k i · 2 i ( k + k i · 2 i )conditioned on event E γ · 2 − 2 h · k | Ω | · k · k i · 2 i ( k + k i · 2 i )conditioned on event E for an absolute constant γ .
Proof.
We first observe that since the g p are standard normal Gaussians, the entire estimator is
Gaussian distributed with variance
[Equation]
We have ( v S , 2 − ( h +1) p − v S , 2 − h p ) =
( v S , 2 − ( h +
[Equation]
p − cost ( p, S )
+ cost ( p, S )
− v S , 2 − h p ) ≤ 2 · 2 − h · (cid:112) cost ( p, A ) cost ( p, S ) due to Lemma 4.
Furthermore, by definition w p
=
cost ( P, A )
| C j | cost ( C j , A ) .
Finally, by definition of type i , we have cost ( p, S ) · | C j | =
O (1) · cost ( C j , S ) and by Assumption 4 we have cost ( p, A ) · | C j | = O (1) · cost ( C j , A ) for all p ∈ C j .
(cid:88)
[Equation]
T i
[Equation]
[Equation]
cost
[Equation]
A
[Equation]
[Equation]
A
[Equation]
S )
[Equation]
A
[Equation]
(cid:33)
Now, let k i be the number of clusters of type i .
Then due to Assumption 4 cost ( C j , S ) · k i ≤
O (1) cost ( P, S ) , for all C j of type i .
Finally, note that cost ( P, A ) cost ( C j , A ) ≤
O (1) · k , also due to Assumption 4.
Combining this, we then have
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table] [Equation] [Table] [Equation]
[Table]
[Equation]
Assuming event E , this may now be bounded by O (1) · 2 − 2 h | Ω | · k
[Equation]
If event E does not hold, we may bound the term by 2 − 2 h · k k
[Equation]
Ω | ·
[Equation]
Completing the Proof (for Eq. 7)
Throughout this section, we use the bound on the expected maximum of independent Gaussians.
Lemma 8 (Lemma 2.3 of Massart [2007]) .
Let g i ∼ N (0 , σ 2 i ) , i ∈ [ n ] be Gaussian random variables and suppose σ i ≤ σ for all i .
Then E [max i ∈ [ n ] | g i | ] ≤ 2 σ · √ 2 ln n.
The number of cost vectors in
N h
+ ×
N h is at most exp (cid:0) γ · k · log ∥
P ∥ 0 · min( k i + 2 2 h , 2 2 h · 2 i ) · i · h ) (cid:1) for some absolute constant γ due to
Lemma 4.
With the bound on the variance (Lemma 7 and conditioned on event E ), we then have log ε −
[Equation]
(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) E  
[Equation]
Conditioned on event E not holding, we then have using a similar calculation
[Equation]
(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:80) [Equation] − v [Equation]
w
[Equation]
[Equation] P
[Equation]
(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) E  
[Equation]
We have P [ E ] ≤ 1 /k 2 due to Lemma 6.
Since ∥ P ∥ 0 ≤ poly ( k, ε − 1 ) , 2 i ≤ O (1) · ε − 2 and by our choice of | Ω | , we can combine Equations 10 and 11 with the law of total expectation to obtain
[Equation]
(cid:12)(cid:12)(cid:12)(cid:12) (cid:12)(cid:12)  
[Equation]
Remark 2. The proof did not rely on properties of squared Euclidean distances. The only thing that is necessary to show is that the nets have the error as required by Defintion 1.
Thus, the same proof and the same improved coreset bound also holds for k -median.
Disclosure of Funding Acknowledgements
Kapser Green Larsen was partially supported by the Independent Research Fund Denmark (DFF) under a Sapere Aude Research Leader grant No 9064-00068B.
David Saulpic has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (Grant agreement No.
“The Design of Modern Fully Dynamic Data
Structures (MoDynStruct)”.
Chris Schwiegelshohn was partially supported by the Independent Research Fund Denmark (DFF) under a Sapere Aude Research Leader grant No 1051-00106B and the Innovation Fund Denmark under grant agreement No 0153-00233A.
Omar Ali Sheikh-Omar was partially supported by the Innovation Fund Denmark under grant agreement No 0153-00233A
A Analysis Details A.1 Omitted Proofs (Details for Lemma 3)
Clustering Nets Next, we give a detailed proof of Lemma 4.
Proof of Lemma 4.
Our objective is to generate a small set of cost vectors that satisfy the desired guarantee.
We first define the cost vectors (the reader familiar with the proof sketch from the main body of the submission may skip this and the next paragraph).
For each subset U of size O (min( α − 2
i , α − 2 + k i ) , we consider the the subspace
Π U spanned by U .
In this subspace we consider ( α/ 2 i ) · (cid:112) cost ( p, A ) - nets of every ball centered around Π U p with radius 60 · 2 i / 2 · (cid:112) cost ( p, A ) for all p ∈ P .
Such a net has size exp( γ · rank ( U ) i log α ) , for some constant γ and there exist at most
∥ P ∥ 0 · exp( γ ·| U | i log α )
many such nets.
Furthermore, there are at most (cid:0)
∥
P ∥ 0 | U | (cid:1) =
∥
P ∥
| U | 0 such subsets.
Now, for every point p , define an exponential sequence α 2 (1 + α/ 2 i ) j for j ∈ { 0 , . . .
log 102 i } .
There exist at most ∥ P ∥ 0 such sequences and every such sequence consists of at most O ( α − 1 · 2 i · i ) many values.
We combine every net point in ever ball of every subspace with all values in the exponential sequence to obtain the evaluation for a single candidate center.
The overall number of candidate centers is therefore of the order ∥ P ∥ | U |· exp( γ · | U | i log α ) , for a sufficiently large γ .
The overall number of candidate cost vectors is now the number of k subsets of candidate centers, i.e. ∥ P ∥ k ·| U |· exp( γ · k · | U | i log α ) .
Combined with the bounds on U , this yields the desired size.
What remains to be shown is that the thus constructed cost vectors satisfy the guarantee of a ( α, k ) -clustering net.
We start by noting that since all points are from clusters of type i with i ≥ 3 , we have
[Equation]
as a straightforward application of the triangle inequality.
To simplify the calculations, we use the bound cost ( p, S ) ≤ 2 i cost ( p, A ) ; the result then follows by folding in the constant factor in the definition of α .
Let S be a candidate solution and let S ′ be the subset of S serving the points of P .
For a center s ∈ S ′ , let U 1 be the subset of vectors of P given by Lemma 5 with ε := α · 2 i/ 2 and let Ube the subset of vectors of P given by Lemma 5 with ε :=
α and let U ′
=
U 2 ∪ A .
Let U = (cid:26)
Uif rank ( U 1 ) ≤ rank ( U ′ 2 )
U ′else and let Π U be the orthogonal matrix of the span of U .
Suppose, for every point p in the span of U , we are given an α/ (4 · 2 i ) · (cid:112) cost ( p, A ) -net of the
Euclidean ball centered around Π U p with radius 60 · 2 i / 2 · (cid:112) cost ( p, A ) .
Such a net has size exp( γ · rank ( U ) i log α ) , for some constant γ and there exist at most
∥ P ∥ 0 · exp( γ · rank ( S )
i log α ) many such nets.
Denote by s ′ the point in these nets closest to Π U s .
We now argue that either the cost is preserved for any given point, or that the net determines that this point costs significantly more than than a factor 2 i cost ( p, A ) .
Specifically, we argue that
[Equation]
Π
[Equation]
p
[Equation]
p
[Equation]
A ) .
Let
p
∈
P be the point with minimal cost ( p, A ) satisfying the first condition.
Then for any point p that also satisfies this condition ∥
Π U s − s ′
∥ ≤ α 4 · 2 − i 2 i/ 2 · (cid:112) cost ( p ′ , A ) ≤ α 4 · 2 − i/ 2 (cid:112) cost ( p ′ , A ) ≤ α 4 · 2 − i/ 2 (cid:112) cost ( p, A ) .
We then have
[Equation]
Conversely, suppose p satisfies the second condition.
Then ∥ Π U ( p − s ) ∥ ≥ 20 · 2 i / 2 (cid:112) cost ( p, A ) .
Let q be the point in the ( α/ 2 i ) -net of the ball centered around Π U p with radius 20 · 2 i closest to Π U s .
This implies ∥ q − Π U p ∥ ≥ 20 · 2 i / 2 · (cid:112) cost ( p, A ) − ( α/ 2 i ) · (cid:112) cost ( p, A ) ≥ 10 · 2 i / 2 · (cid:112) cost ( p, A ) .
Next, we consider a net over all values ∥
( I − Π U Π TU ) s ∥ .
For every point p , define an exponential sequence α · 2 − i/ 2 (1 + α/ (4 · 2 i ))
j · (cid:112) cost ( p, A ) for j ∈ { 0 , . . .
log(10 α − 2 · 2 i ) · 8 · 2 i α } .
There exist at most ∥ P ∥ 0 such sequences and every such sequence consists of at most O ( α − 1 · 2 i · i log α ) many values.
Similar to the bound above, we will show that there exists an s ′′ in the union of sequences such that
[Equation] [Equation]
Observe that the point p ′ closest to s is contained in U .
This implies for all p ∈ P that ∥
( I − Π U Π TU ) s ∥ ≤ (cid:112) cost ( p ′ , s ) ≤ (cid:112) cost ( p, s ) .
Now, any two successive elements of the sequence have a ratio of (1 + α/ (4 · 2 i )) .
Hence, we have (cid:12)(cid:12) s ′′ − ∥
( I − Π U Π TU ) s ∥ (cid:12)(cid:12) ≤ ∥
[Equation]
I
[Equation] [Equation]
∥ · α/
[Equation]
4 · 2 i )
[Equation]
α (cid:112) cost ( p ′ , A ) ≤
[Equation]
/ (4 · 2 i
[Equation]
≤ α 4 · 2 − i/ 2 (cid:112) cost ( p, A ) .
Thus
[Equation]
We now conclude for a single center s .
Assume that the point in P closest to s is included in U .
Let s ′ be the point in the span of U satisfying Eq.
and let s ′′ be the element in the exponential sequence satisfying Eq.
We wish to show that for some absolute constant β
[Equation]
First
[Equation]
we decompose ∥ p − s ∥ 2 as follows.
We have
[Equation]
We first focus on the case that ∥ p − s ∥ ≤ 60 · 2 i / 2 (cid:112) cost ( p, A ) .
Since projection only decreases the norm, this implies that ∥ Π U ( p − s ) ∥ and ∥ ( I − Π U Π TU )( p − s ) ∥ are less than 60 · 2 i / 2 (cid:112) cost ( p, A ) as well.
This implies that Equations 13 and 15 hold, which implies
[Equation]
A ) + | 2 p T
[Equation]
I − Π U Π TU ) s |
Since the point closest to s is contained in U , we have ∥ ( I − Π U Π TU ) s ∥ ≤ (cid:112) cost ( p, s ) .
Suppose | U | =
α − 2 · 2 i .
Then Lemma 5 gives
[Equation]
This implies
[Equation]
and hence
[Equation]
as desired.
Now suppose α − 2 + k i .
In this case ∥
[Equation]
Consequently,
[Equation]
as desired.
We now focus on the case that ∥ p − s ∥ ≥ 60 · 2 i/ 2 (cid:112) cost ( p, A ) .
In this case, either ∥ Π U ( p − s ) ∥ or ∥ ( I − Π U Π TU )( p − s ) ∥ are at least 40 · 2 i / 2 (cid:112) cost ( p, A ) .
In the former case, Eq.shows that we rule out s as a viable center for p .
In the latter case, we have due to Lemma 5 ∥
[Equation]
I
[Equation] [Equation]
=
[Equation]
≥
[Equation]
1 − α ) · (cid:0) ∥
( I − Π U Π TU ) p ∥ 2 + ∥
( I − Π U Π TU ) s ∥ 2 (cid:1) .
Here, either ∥ ( I − Π U Π TU ) p ∥ or ∥ ( I − Π U Π TU ) s ∥ are now at least 20 · 2 i / 2 (cid:112) cost ( p, A ) .
But the latter cannot be true, as ∥ ( I − Π U Π TU ) s ∥ is less than the distance of s to its closest point.
Therefore simply evaluating ∥ ( I − Π U Π TU ) p ∥ rules out s as a viable center.
We can now conclude overall.
By assumption, all points were members of clusters of type i .
Therefore, there exists at least one viable center for every point.
Thus only the upper equation in 15 applies.
The overall lemma now follows by rescaling α .
Remark 3.
The same net construction essentially also works for k -median.
There are essentially only two differences.
First cost ( p, S ) =
min s ∈S ∥ p − s ∥ and thus the value of i in cost ( p, S ) cost ( p, A )
≈
i is the square root of what it was before.
So overall, we have nets of size exp(min( α − 2 · 2 − i/ 2 , α −
+ k i )) .
Second, when bounding
[Equation]
we observe that the term on the right is at most ∥ p − s ∥ · (cid:16) 1 ± α · 2 i/ 2 cost ( p, A ) cost ( p, S ) (cid:17) 1 / 2 .
Thus, the difference in cost from our nets and the true cost is at most
O ( α · (cid:112) cost
[Equation]
p, A ) cost ( p, S )) .
We also require an additional net that works for low dimensions.
Lemma 9 (Compare Lemma 22 of Cohen-Addad et al.
Let P be a set of points in d dimensional Euclidean space, k a positive integer and A be a candidate solution.
Define C to be the set of possible candidate centers such that the clusters induced by A are of type i , with 3 ≤ i ≤ log 1 /ε 2 .
For all α ≤ 1 / 2 , there exists an ( α, k ) -clustering net N of C with
[Equation]
where γ is an absolute constant.
Proof.
The only difference to Lemma 22 of Cohen-Addad et al.
[2022] is that the nets are required to have an error of α · (cid:112) cost ( p, S ) cost ( p, A ) rather than α · ( cost ( p, S )
+ cost ( p, A )) .
This can be done by rescaling ε by 2 − i , which in turn is absorbed by the constant γ as 2 i ≤ O (1) · ε − 2 .
Variance Bounds
Controlling Eq.
Here, we use Lemma 9 and Assumption 2 to show that the number of cost vectors in N h +
× N h is at most exp (cid:0) γ · k · log ∥ P ∥ 0 · ε − 2 log h/ε ) (cid:1) .
Conditioned on event E , we therefore have
[Equation]
C j ∈ T i (cid:80)
[Equation]
( v
[Equation]
−
[Equation]
( cost (
[Equation]
g p (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) E   ≤
[Table]
[Equation]
log
[Equation]
[Table]
[Equation]
Similarly, if E does not hold, we have
[Equation]
E  
[Equation]
We have P [ E ] ≤ 1 /k 2 due to Lemma 6.
Since ∥ P ∥ 0 ≤ poly ( k, ε − 1 ) , 2 i ≤ O (1) · ε − 2 and by our choice of | Ω | , we can combine the last two equations with the law of total expectation to obtain ∞
[Equation]
≤
[Equation]
Observe that Eq.
16 and Eq.
12 are essentially identical up to lower order terms.
Controlling Eq.
Here, we first split Eq.
6 into two estimators that will be easier to handle.
We split the estimator into two parts as follows.
First, let q j := (cid:80)
p ∈
Cj v S , 2 − 1 p | C j | .
Now we consider
[Equation]
Thus
[Equation]
Equation 6 becomes
[Equation]
+
[Equation] [Equation]
(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:35) =
[Equation]
(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:80)
[Equation]
|
[Equation]
( P
[Equation]
A )
[Equation]
(cid:12) (cid:12)(cid:12)(cid:12)(cid:12) (cid:35) (19)
[Equation]
(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) (cid:80)
[Equation]
| Ω | · ( cost ( P, S ) + cost ( P, A ))
g p (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:35) (20)
Due to Assumption 4, we have cost ( T i , S )
=
O (1) · k i cost ( C j , S ) , for any C j ∈ T i
Thus
E Ω E g (cid:34) sup v S , 1 ∈ N 2 −(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) (cid:80) C j ∈ T
i (cid:80) p ∈
C
j ∩ Ω q j · w p | Ω | · ( cost ( P, S )
+ cost ( P
[Equation]
A ))
g p (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:35) ≤
[Equation]
(cid:34)
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation] [Equation]
For the variance of the estimator used for Eq.
19, we use the following lemma.
Lemma 10.
If Assumption 4 holds, the variance of 1 | Ω | (cid:80)
Cj ∈
Ti (cid:80) p
∈
Cj ∩ Ω ( v S , 2 − 1 p − q j ) w p ( cost ( P, S )
+ cost ( P, A ))
g p is at most γ · 1 | Ω | · k · k i · 2 i ( k + k i · 2 i )conditioned on event E γ · k | Ω | · k · k i · 2 i ( k + k i · 2 i )conditioned on event E for an absolute constant γ .
Proof.
We will bound | v S , 2 − 1 p − q j | for any point p ∈ C j .
Due to the triangle inequality and by
Assumption 4 which states that all points have roughly equal distance to their center in A , we have
[Equation]
Futhermore, again due to the triangle inequality, C j ∈ T i with i > 3 and Assumption 4, we have (
[Equation]
cost ( p, S .
Therefore
[Equation]
Using this bound and the same steps as in Lemma 7, we then have
[Equation]
T i (cid:88) p ∈
C
j ∩ Ω | Ω |cost ( p, S ) cost ( p, A ) cost ( P, S ) 2 | C j |cost ( C j , A ) 2 · ( cost ( P, S )
+ cost ( P, A ))
≤ O (1) · (cid:88)
C j ∈ T i (cid:88)
p ∈ C
j ∩ Ω (cid:18)
k k i · | Ω |(cid:19) · k · k i · 2 i ( k + k i · 2 i )
Conditioned on event E , this now becomes O (1) · 1 | Ω | · k · k i · 2 i ( k + k i · 2 i ) 2 and similarly, if event E does not hold, we have the bound k | Ω | · k · k i · 2 i ( k + k i · 2 i ) 2 .
Remark 4.
Here, there is a minor difference in the proof of k -median.
We could only apply the triangle inequality for (cid:113) v S , 2 − 1 p − √ q j , as we were dealing with sum of distances.
For k -median we would apply apply the triangle inequality directly to v S , 2 − 1 p − q j .
We now focus on the variance of the estimator used for Eq. 20.
Due to Assumption 4, we have cost ( T i , S )
=
O (1) · k i cost ( C j , S ) , for any C j ∈ T i
Thus
[Equation]
sup
[Equation]
We now obtain the following variance for the estimator used in Equation 22.
Lemma 11.
If Assumption 4 holds, the variance of (cid:80) p ∈
Cj ∩
Ω q j · w p | Ω |· cost ( C j , S ) g p , given that C j ∈ T i with i ∈ { 3 , . .
. , log ε − 2 } is at most
[Equation]
conditioned on event E γ
[Equation]
k| Ω | conditioned on event E for an absolute constant γ .
Proof.
Recall by Assumption 4 cost ( P, S )
=
O (1) · k · cost ( C j , S ) .
We have
[Equation]
Conditioned on event E , | C j ∩ Ω | = 1 k · | Ω | and this now becomes O (1) · k | Ω | .
Otherwise, we have the bound k 2 | Ω | .
We now bound Equations 19 and 22.
For the former, we have | N 2 − 1 | ≤ exp
[Equation]
Thus, combined with Lemma 10 and conditioning on event E , we have
[Equation]
Similarly, not conditioning on event E implies
[Equation]
We have P [ E ] ≤ 1 /k 2 due to Lemma 6.
Plugging in ∥ P ∥ 0 ≤ poly ( k, ε − 1 ) and our choice of | Ω | , we can combine the last two equations with the law of total expectation to obtain
[Equation]
For the term in Equation 22, we note that q
[Equation] [Equation]
p cost
[Equation]
C j , S )
= cost ( P, A ) .
Thus, for every cluster, we have a net of size 1 , which means we have an overall net of size k .
We thus obtain
[Equation]
Similarly, conditioning on event E implies
[Equation]
Combining both terms, using P [ E ] ≤ 1 /k 2 due to Lemma 6 and the law of total expectation, we obtain
[Equation]
Combining the bounds in Equations 12, 16, 23 and 24 for the respective terms in Equations 7, 8,and 22 now yields the claim.
B Experimental Results
We complement our results with an empirical evaluation of coreset algorithms.
Our aim is to only study whether a linear dependency on k is (empirically) the correct answer or not.
For a more complete experimental study on coresets, including the sampling distributions studied here, we refer to Schwiegelshohn and Sheikh-Omar [2022].
Specifically, we ran the algorithm used in our analysis (Group Sampling) as well as the popular Sensitivity Sampling ( ? ) and uniform sampling on synthetic and real world data.
Here, we mainly focus on the performance of the three real world instances Census 4 , Covertype 5 and Tower 6 .
It is hard to compute the distortion of a given coreset, so for the real world data sets, we sampled | Ω | = 200 k points from each distribution, followed by running k -means++
(see Arthur and Vassilvitskii [2007]) 10 times and compared the costs, as well as the distortion on the found solution (see Fig. 6).
Every coreset was computed 10 times as well.
The distortion is the maximum distortion over each run among all evaluated solutions.
In terms of cost, all algorithms computed solutions of roughly equal value.
In terms of distortion, uniform sampling expectedly had the worst bounds.
Somewhat interestingly, the Group Sampling algorithm which currently has a better worst case bound than Sensitivity Sampling tends to have a worse distortion.
We leave it to future work to find an analysis of Sensitivity Sampling that matches (or outperforms) Group Sampling.
Furthermore, there is a small increase in distortion with increasing k .
While this suggests that a linear dependency in k is not the correct answer, the increase in distortion is too small to suggest that a k 1+ δ dependency for some δ > 0 will be the correct answer.
It may also be too difficult to accurately measure distortion on real world instances as uniform sampling, which is known to be an extremely bad coreset algorithm in theory, performs moderately competitively.
[Figure]
Synthetic Data Sets The first synthetic instance, henceforth called Instance 1 , is simply the hard instance from Cohen-Addad et al.
[2022], which are just m standard unit vectors in R m .
CohenAddad et al.
[2022] showed that for m ∈ Ω( k/ε 2 ) , no significant space savings can be achieved.
We varied m for additional values m = { k 1 .
25 /ε 2 , k 1 .
5 /ε 2 } with ε ∈ { 0 . 2 , 0 . 1 , 0 .
and k ∈ { 10 , 20 , 50 , 70 , 100 } .
The second synthetic instance, henceforth called Instance 2 , is constructed as follows.
We select k/ 2 standard basis vectors.
Around each of the standard basis vectors, we sampled m = { 1 /ε 2 , k 0 .
25 /ε 2 , k 0 .
5 /ε 2 } random vectors with norm ε , with ε ∈ { 0 . 2 , 0 . 1 , 0 . 05 , 0 .
and k ∈ { 10 , 20 , 50 , 70 , 100 } .
The reason for choosing this data set is because this is the arguably simplest instance for which we could not give an improve the ˜ O
( k 1 . 5 /ε 2 ) analysis given in this paper.
For these data sets, Group Sampling and Uniform Sampling are identical, which is why we only use one in the evaluations.
Evaluations We selected coresets of sizes proportionate to k/ε 2 , k 1 .
25 /ε 2 and k 1 .
5 /ε 2 .
For every size, we generated 10 coresets.
Subsequently, we generated 100 candidate solutions for every designated coreset, and measured the distortion, i.e. the maximum ratio between coreset cost and original cost (resp.
original cost and coreset cost, if the coreset cost became cheaper).
For each instance and every combination of the parameters, we gave a box plot of the distortions and, for completeness, the costs of the algorithm when running k -means++ and returning the cheapest one.
Results For both synthetic data sets, we observed that no matter how the hard instance was constructed, the distortion measured on the evaluated instances decreased with increasing k .
Since that was the case even for coresets of size O ( k/ε 2 ) , this suggests that the dependency on k is non-increasing.
Between Sensitivity Sampling and Group Sampling/Uniform Sampling, the latter distribution usually obtained slightly better distortions, though overall the two algorithms seem to behave similarly.
We conjecture that a both should yield a coreset when sampling ˜ O ( k/ε 2 ) many points.
B.1 Synthetic InstanceCosts on the Instance 1 ( ϵ = 0 . 01 )
[Table]
3.499e+02) 5.303e+06 (3.202e+02) k 1 .
/ 10 ϵ8.785e+06 (2.752e+02) 8.785e+06 (2.729e+02) k 1 .
25 / 10 ϵ8.785e+06 (2.841e+02) 8.785e+06 (2.570e+02) k 1 .
50 / 10 ϵ8.785e+06 (2.550e+02) 8.785e+06 (2.361e+02) k 1 .
/ 10 ϵ1.500e+07 (2.222e+02) 1.500e+07 (2.139e+02) k 1 .
25 / 10 ϵ1.500e+07 (2.095e+02) 1.500e+07 (2.456e+02) k 1 .
50 / 10 ϵ1.500e+07 (2.442e+02) 1.500e+07 (2.230e+02)
[Table]
k 1 . 00 / 10 ϵ1.896e+04 (3.079e+01) 1.896e+04 (2.783e+01) k 1 . 25 / 10 ϵ1.896e+04 (2.824e+01) 1.896e+04 (2.732e+01) k 1 .
50 / 10 ϵ1.896e+04 (2.666e+01) 1.896e+04 (2.891e+01) k 1 . 00 / 10 ϵ5.363e+04 (1.959e+01) 5.363e+04 (2.021e+01) k 1 .
25 / 10 ϵ5.363e+04 (2.117e+01) 5.363e+04 (2.121e+01) k 1 .
50 / 10 ϵ5.363e+04 (1.959e+01) 5.363e+04 (2.060e+01) k 1 .
/ 10 ϵ2.120e+05 (1.292e+01) 2.120e+05 (1.313e+01) k 1 .
25 / 10 ϵ2.120e+05 (1.325e+01) 2.120e+05 (1.282e+01) k 1 .
50 / 10 ϵ2.120e+05 (1.323e+01) 2.120e+05 (1.363e+01) k 1 .
/ 10 ϵ3.513e+05 (1.121e+01) 3.513e+05 (1.102e+01) k 1 .
25 / 10 ϵ3.513e+05 (1.087e+01) 3.513e+05 (1.137e+01) k 1 .
50 / 10 ϵ3.513e+05 (1.105e+01) 3.513e+05 (1.174e+01) k 1 .
/ 10 ϵ5.998e+05 (9.413e+00) 5.998e+
(9.632e+00)
k 1 . 25 / 10 ϵ5.998e+05 (9.848e+00) 5.998e+
(9.281e+00)
k 1 .
50 / 10 ϵ5.998e+05 (9.161e+00) 5.998e+05 (9.166e+00)
[Figure]
[Figure] [Figure]
Costs on the Instance 1 ( ϵ = 0 . 1 )
[Table]
k
[Table] [Table]
[Table]
+03 (6.
[Table]
[Table] [Table]
00) k
[Table]
25 / 10 ϵ
[Table]
4.727e+03 (7.
[Table] [Table]
+03 (
[Table] [Table]
00) k
[Table]
50 / 10 ϵ4.
[Table]
[Table] [Table]
[Table] [Table]
k
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table]
k
[Table] [Table]
ϵ
[Table] [Table]
(5.094e
[Table] [Table]
[Table]
k
[Table] [Table]
ϵ
[Table] [Table]
(5.013e
[Table] [Table]
[Table] [Table]
[Table] [Table]
k
[Table] [Table]
[Table] [Table]
[Table] [Table]
k
[Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
.890e
[Table]
00) k
[Table] [Table]
5.294e
[Table] [Table]
.941e+00) 5.
[Table] [Table]
[Table] [Table]
k
[Table] [Table]
ϵ
[Table] [Table]
+
[Table]
(3.781e+00) 8.
[Table] [Table]
[Table]
k
[Table] [Table]
[Table]
+04 (3.
[Table]
+00) 8.
[Table]
+
[Table] [Table]
[Table]
+00) k
[Table] [Table]
ϵ
[Table]
8.772e+
[Table]
(3.828e+00) 8.772e+04 (3.839e+00)
[Table]
k 1 . 00 / 10 ϵ1.498e+05 (3.988e+00) 1.498e+05 (3.786e+00)
k 1 .
25 / 10 ϵ1.498e+05 (4.011e+00) 1.498e+05 (3.943e+00) k 1 .
50 / 10 ϵ1.498e+05 (4.014e+00) 1.498e+05 (3.955e+00)
[Table]
k
[Table]
/ 10 ϵ1.168e+03 (2.158e+00) 1.168e+03 (2.223e+00) k 1 .
25 / 10 ϵ1.168e+03 (2.153e+00) 1.168e+03 (2.170e+00)
k 1 .
50 / 10 ϵ1.168e+03 (2.166e+00) 1.168e+03 (2.095e+00)
[Table]
k 1 . 00 / 10 ϵ3.319e+03 (2.065e+00) 3.319e+03 (2.096e+00) k 1 .
25 / 10 ϵ3.319e+03 (2.088e+00) 3.319e+03 (2.063e+00)
k 1 .
50 / 10 ϵ3.319e+03 (2.077e+00) 3.319e+03 (2.141e+00)
[Table]
k 1 . 00 / 10 ϵ1.317e+04 (2.446e+00) 1.317e+04 (2.415e+00) k
[Table]
25 / 10 ϵ1.317e+04 (2.453e+00) 1.317e+04 (2.379e+00) k 1 .
50 / 10 ϵ1.317e+04 (2.374e+00) 1.317e+04 (2.397e+00)
k 1 . 00 / 10 ϵ2.184e+04 (2.733e+00) 2.184e+04 (2.643e+00)
k 1 .
25 / 10 ϵ2.184e+04 (2.730e+00) 2.184e+04 (2.650e+00) k 1 .
50 / 10 ϵ2.184e+04 (2.657e+00) 2.184e+04 (2.727e+00) k 1 . 00 / 10 ϵ3.732e+04 (2.978e+00) 3.732e+04 (3.016e+00) k 1 .
25 / 10 ϵ3.732e+04 (2.987e+00) 3.732e+04 (2.970e+00) k 1 .
50 / 10 ϵ3.732e+04 (3.058e+00) 3.732e+04 (3.032e+00)
[Table]
Sampling k 1 .
/ 10 ϵ1.0000 (
[Table] [Table]
[Table]
k 1 . 25 /
[Table]
ϵ1.0000 (
[Table] [Table]
k 1 .
50 / 10 ϵ1.0000 (0.00003) 1.0000 (0.00003) k 1 . 00 / 10 ϵ1.0000 (0.00001) 1.0000 (0.00001) k 1 .
25 / 10 ϵ1.0000 (0.00002) 1.0000 (0.00002) k 1 .
50 / 10 ϵ1.0000 (0.00001) 1.0000 (0.00001) k 1 . 00 / 10 ϵ1.0000 (0.00001) 1.0000 (0.00001) k 1 .
25 / 10 ϵ1.0000 (0.00001) 1.0000 (0.00001) k 1 .
50 / 10 ϵ1.0000 (0.00000) 1.0000 (0.00000) k 1 . 00 / 10 ϵ1.0000 (0.00001) 1.0000 (0.00001) k 1 .
25 / 10 ϵ1.0000 (0.00001) 1.0000 (0.00001) k 1 .
50 / 10 ϵ1.0000 (0.00000) 1.0000 (0.00000) k 1 . 00 / 10 ϵ1.0000 (0.00001) 1.0000 (0.00001) k 1 .
25 / 10 ϵ1.0000 (0.00000) 1.0000 (0.00000) k 1 .
50 / 10 ϵ1.0000 (0.00000) 1.0000 (0.00000)
[Table]
k 1 .
25 / 10 ϵ1.0004 (0.00048)
[Table] [Table]
k
[Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
k 1 .
/ 10 ϵ1.0003 (0.00034) 1.0003 (0.00033)
k 1 .
25 / 10 ϵ1.0002 (0.00020) 1.
[Table] [Table]
k
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table]
k
[Table] [Table]
[Table] [Table]
k 1 .
/ 10 ϵ1.0026 (0.00123) 1.0027 (0.00116) k 1 .
25 / 10 ϵ1.0015 (0.00192) 1.0014 (0.00195)
k 1 .
50 / 10 ϵ1.0010 (0.00123) 1.0010 (0.00122) k 1 . 00 / 10 ϵ1.0013 (0.00130) 1.0013 (0.00132) k 1 .
25 / 10 ϵ1.0007 (0.00080) 1.0006 (0.00079) k 1 .
50 / 10 ϵ1.0004 (0.00050) 1.0004 (0.00048) k 1 . 00 / 10 ϵ1.0009 (0.00106) 1.0009 (0.00104)
k 1 . 25 / 10 ϵ1.0005 (0.00059) 1.0005 (0.00060)
[Table]
[Table]
B.2 Synthetic InstanceCosts on the Instance 2 ( ϵ = 0 . 01 )
[Table]
k
[Table] [Table]
[Table]
532e+09 (
[Table]
+09)
[Table]
k
[Table] [Table]
ϵ
[Table]
09 (2.514e
[Table] [Table]
k
[Table]
50 / 10 ϵ7.533e+09 (2.516e+09) 7.531e+09 (2.517e+09)
[Table]
k 1 . 00 / 10 ϵ2.489e+10 (3.562e+09) 2.489e+10 (3.561e+09) k 1 .
25 / 10 ϵ2.490e+10 (3.560e+09) 2.137e+10 (3.559e+09) k 1 .
50 / 10 ϵ2.488e+10 (3.562e+09) 2.137e+10 (3.561e+09) k 1 . 00 / 10 ϵ1.070e+11 (5.644e+09) 1.070e+11 (5.651e+09) k 1 .
25 / 10 ϵ1.070e+11 (5.643e+09) 1.070e+11 (5.641e+09) k 1 .
50 / 10 ϵ1.070e+11 (5.637e+09) 1.070e+11 (5.642e+09) k 1 . 00 / 10 ϵ1.867e+11 (6.677e+09) 1.867e+11 (6.672e+09) k 1 .
25 / 10 ϵ1.867e+11 (6.678e+09) 1.867e+11 (6.675e+09)
k 1 .
50 / 10 ϵ1.867e+11 (6.677e+09) 1.867e+11 (6.679e+09) k 1 . 00 / 10 ϵ3.270e+11 (7.990e+09) 3.269e+11 (7.990e+09) k 1 .
25 / 10 ϵ3.270e+11 (7.998e+09) 3.270e+11 (8.015e+09)
k 1 .
50 / 10 ϵ3.269e+11 (7.992e+09) 3.270e+11 (7.987e+09)
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
.749e
[Table] [Table]
[Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
.740e
[Table] [Table]
.127e
[Table]
514e
[Table]
126e
[Table]
517e
[Table]
127e
[Table]
127e
[Table] [Table]
[Table] [Table]
[Table] [Table]
ϵ1.0710 (0.09212) 1.0631 (0.08432) k 1 .
25 / 10 ϵ1.0502 (0.06418) 1.0457 (0.05854) k 1 .
50 / 10 ϵ1.0328 (0.04265) 1.0278 (0.03545) k 1 . 00 / 10 ϵ1.0320 (0.03946) 1.0296 (0.03859) k 1 .
25 / 10 ϵ1.0197 (0.02511) 1.0190 (0.02300) k 1 .
50 / 10 ϵ1.0116 (0.01466) 1.0106 (0.01364) k 1 . 00 / 10 ϵ1.0237 (0.02871) 1.0236 (0.02887) k 1 .
25 / 10 ϵ1.0136 (0.01707) 1.0129 (0.01588) k 1 .
50 / 10 ϵ1.0080 (0.00992) 1.0077 (0.00906) k 1 . 00 / 10 ϵ1.0184 (0.02253) 1.0177 (0.02165) k 1 .
25 / 10 ϵ1.0095 (0.01185) 1.0094 (0.01151)
[Table]
B.3 Real-world Data Sets
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table]
1.02e+00 (8.56e-03) 1.01e+00 (6.83e-03) 1.03e+00 (2.16e-02)
1.02e+00 (9.85e-03) 1.01e+00 (5.55e-03) 1.02e+00 (1.40e-02)
1.02e+00 (5.66e-03)
1.00e+00 (4.14e-03)
[Table]
Distortions on the Covertype
[Table]
[Table] [Table]
[Figure]
[Figure] [Figure]
[Figure] [Figure]